1. Field of the Invention
The present invention is directed to the collection of market statistics from a network.
2. Background Art
“Web crawlers”, “robots”, or “spider engines” are programs used to automatically search the Internet for web pages or documents of interest. The information found by the spider engine may be collected, cataloged, and otherwise used by search engines. For example, a spider engine may be directed to search for and collect particular types of data, such as product catalog information, or may randomly search and catalog all found web pages to create a web index. The spider engine may enter a particular web site, and search one or more web pages of the web site for information of interest.
Spider engines can collect a large amount of information. This information is difficult to analyze. Accordingly, there is a need for a method, system, and computer program product that generates statistics from data collected from a network.